Solid-state imaging devices such as a CCD image sensor or a complementary metal oxide semiconductor (CMOS) image sensor that are used in a video camera or a digital still camera perform photoelectric conversion for accumulating a charge according to an amount of incident light and outputting an electric signal corresponding to the accumulated charge. However, there is an upper limit in a charge accumulation amount in a photoelectric conversion element. If light of an amount equal to or more than a constant amount is received, so-called overexposure in which an accumulation charge amount reaches a saturation level and a brightness level of a subject region of constant brightness or more is set to a saturated brightness level may be generated.
To prevent such a phenomenon, a process for controlling a charge accumulation period in the photoelectric conversion element according to a change of external light, adjusting an exposure time, and controlling sensitivity to an optimal value is executed. For example, for a bright subject, a shutter is released fast to decrease an exposure time and decrease the charge accumulation period in the photoelectric conversion element and the electric signal is output before the accumulation charge amount reaches the saturation level. By this process, an output of an image in which a gradation according to a subject is exactly reproduced is enabled.
However, if the shutter is released fast in imaging of a subject in which a bright place and a dark place are mixed, a sufficient exposure time is not taken in a dark portion. For this reason, S/N is deteriorated and an image quality is lowered. As such, in an image obtained by imaging the subject in which the bright place and the dark place are mixed, exactly reproducing bright levels of a bright portion and a dark portion requires a process for increasing an exposure time and realizing high S/N in pixels in which incident light on an image sensor is small and avoiding saturation in pixels in which incident light is large.
As a method for realizing such a process, a method of continuously imaging a plurality of images having different exposure times and combining the plurality of images is known. That is, this method is a method of continuously and individually imaging a long time exposure image and a short time exposure image, executing a combination process using the long time exposure image for a dark image region and using the short time exposure image for a bright image region in which overexposure may be generated in the long time exposure image, and generating one image. As such, the plurality of different exposure images are combined, so that an image of a wide dynamic range not having the overexposure, that is, a wide dynamic range image (HDR image) can be obtained.
For example, Patent Document 1 (JP 2000-50151 A) discloses a configuration in which two images to which a plurality of different exposure times are set are imaged, these images are combined, and an image of a wide dynamic range is obtained. This process will be described with reference to FIG. 1. An imaging device outputs image data of two different exposure times in a video rate (30 to 60 fps), for example, when a moving image is imaged. In addition, when a still image is imaged, the imaging device generates image data of two different exposure times and outputs the image data. FIG. 1 is a diagram illustrating characteristics of images (a long time exposure image and a short time exposure image) that are generated by the imaging device and have two different exposure times. A horizontal axis shows a time (t) and a vertical axis shows an accumulation charge amount (e) in a light reception photodiode (PD) configuring a photoelectric conversion element corresponding to one pixel of a solid-state imaging element.
For example, in the case in which a light reception amount of the light reception photodiode (PD) is large, that is, the case of corresponding to a bright subject, as shown in a high brightness region 11 illustrated in FIG. 1, a charge accumulation amount increases rapidly over time. Meanwhile, in the case in which the light reception amount of the light reception photodiode (PD) is small, that is, the case of corresponding to a dark subject, as shown in a low brightness region 12 illustrated in FIG. 1, the charge accumulation amount increases moderately over time.
Times t0 to t3 correspond to an exposure time TL to acquire the long time exposure image. Even when the time is the exposure time TL of the long time, in a line shown in the low brightness region 12, a charge accumulation amount does not reach a saturation level at the time t3 (non-saturation point Py) and an exact gradation expression can be obtained by a gradation level of a pixel determined by using an electric signal obtained on the basis of a charge accumulation amount (Sa).
However, in a line shown in the high brightness region 11, it is clear for the charge accumulation amount to already reach the saturation level (saturation point Px), before reaching the time t3. Therefore, in the high brightness region 11, only a pixel value corresponding to the electric signal of the saturation level is obtained from the long time exposure image. As a result, pixels may become overexposure pixels.
Accordingly, in the high brightness region 11, an accumulation charge of the light reception photodiode (PD) is swept once at a time before reaching the time t3, for example, the time t1 (charge sweeping start point P1) illustrated in the drawing. The charge sweeping is performed to an intermediate voltage holding level controlled in the photodiode (PD), not for the entire charge accumulated in the light reception photodiode (PD). After a charge sweeping process, the short time exposure is performed again at the exposure time TS (t2 to t3). That is, the short time exposure of a period from the short time exposure start point P2 to a short time exposure end point P3 in the drawing is performed. A charge accumulation amount (Sb) is obtained by the short time exposure and a gradation level of a pixel is determined on the basis of an electric signal obtained on the basis of the charge accumulation amount (Sb).
When a pixel value is determined on the basis of the electric signal based on the charge accumulation amount (Sa) obtained by the long time exposure in the low brightness region 12 and the electric signal based on the charge accumulation amount (Sb) obtained by the short time exposure in the high brightness region 251, an estimated charge accumulation amount when the same time exposure is performed and an electric signal output value corresponding to the estimated charge accumulation amount are calculated and a pixel value level is determined on the basis of a calculated result.
As such, the short time exposure image and the long time exposure image are combined, so that an image of a wide dynamic range not having overexposure can be obtained.
However, in all of the configurations described in Patent Document 1, it is necessary to execute the process for individually imaging the long time exposure image and the short time exposure image and combining the long time exposure image and the short time exposure image.
As such, a wide dynamic range image (HDR image) can be generated by using the plurality of images in which the exposure times are changed. However, the following problems occur in the process based on the plurality of images.
Problem 1: imaging needs to be performed several times and a memory to store images needs to be provided.
Problem 2: because a plurality of images of which imaging timings are different are combined or imaging data of long time exposure is used, camera shaking is easily generated.
Also, a process for setting a long time exposure pixel and a short time exposure pixel in one imaging image without using a plurality of images, combining the pixels of different exposure times, and generating a wide dynamic range image has been suggested.
For example, this process is described in Patent Document 2 (JP 11-29880 A) and Patent Document 3 (JP 2000-69491 A).
In this process, setting for arranging a plurality of pixels of the same colors such as R pixels of 2×2 pixels, G pixels of 2×2 pixels, and B pixels of 2×2 pixels in an imaging element (image sensor) is performed, the different exposure times are set to the constituent pixels of the 2×2 pixels of the same colors, and imaging is executed. Pixel values of the same colors of the different exposure times imaged by the image sensor are combined and a wide dynamic range image is obtained.
However, in the configurations described above, noise is likely to be generated due to manufacturing variations of elements or filters and it is difficult to acquire an image of a high quality.